Integration of intra-oral imagery and volumetric imagery

ABSTRACT

Systems and methods are described for identifying a sub-gingival surface of a tooth in volumetric imagery data. Shape data is received from a surface scanner and volumetric imagery data is received from a volumetric imaging device. The shape data of the super-gingival portion of a first tooth is registered with the volumetric imagery data of the super-gingival portion of the first tooth to obtain a registration result. At least one criterion is then determined for detecting a surface of the first tooth in the volumetric imagery data of the super-gingival or the sub-gingival portion using the registration result. The surface of the sub-gingival portion of the first tooth is detected in the volumetric imagery data using the at least one criterion.

RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/847,651, filed on Sep. 8, 2015, which is a continuation of U.S.patent application Ser. No. 13/715,968, filed on Dec. 14, 2012 andentitled “INTEGRATION OF INTRA-ORAL IMAGERY AND VOLUMETRIC IMAGERY,” theentire contents of which are incorporated herein by reference.

FIELD

The disclosure relates to a system, method, and computer readablestorage medium for the integration of intra-oral imagery and volumetricimagery.

BACKGROUND

An intra-oral imaging system is a diagnostic equipment that allows adental practitioner to see the inside of a patient's mouth and displaythe topographical characteristics of teeth on a display monitor. Certainthree-dimensional (3D) intra-oral imagers may be comprised of anintra-oral camera with a light source. The 3D intra-oral imager may beinserted into the oral cavity of a patient by a dental practitioner.After insertion of the intra-oral imager into the oral cavity, thedental practitioner may capture images of visible parts of the teeth andthe gingivae. The 3D intra-oral imager may be fabricated in the form ofa slender rod that is referred to as a wand or a handpiece. The wand maybe approximately the size of a dental mirror with a handle that is usedin dentistry. The wand may have a built-in light source and a videocamera that may achieve an imaging magnification, ranging in scale from1/10 to 40 times or more. This allows the dental practitioner todiscover certain types of details and defects of the teeth and gums. Theimages captured by the intra-oral camera may be displayed on a displaymonitor and may be transmitted to a computational device.

Cone beam computed tomography (CBCT) involves the use of a rotating CBCTscanner, combined with a digital computer, to obtain images of the teethand surrounding bone structure, soft tissue, muscle, blood vessels, etc.CBCT may be used in a dental practitioner's office to generatecross-sectional images of teeth and the surrounding bone structure, softtissue, muscle, blood vessels, etc. During a CBCT scan, the CBCT scannerrotates around the patient's head and may obtain hundreds of distinctCBCT images that may be referred to as CBCT imagery. The CBCT imagerymay be transmitted to a computational device. The CBCT imagery may beanalyzed to generate three-dimensional anatomical data. Thethree-dimensional anatomical data can then be manipulated and visualizedwith specialized software to allow for cephalometric analysis of theCBCT imagery.

SUMMARY

Provided are a system, method, and computer readable storage medium inwhich shape data of a patient's crown and volumetric imagery of thepatient's tooth are received. A determination is made of elements thatrepresent one or more crowns in the shape data. A computational deviceis used to register the elements with corresponding voxels of thevolumetric imagery.

In additional embodiments, a determination is made of volumetriccoordinates and radiodensities corresponding to the voxels.

In further embodiments, at least one of the patient's root is determinedvia region growing from starting locations that include one or more ofthe determined volumetric coordinates and radiodensities at the voxels.

In further embodiments, the region growing is performed by identifyingadjacent voxels that possess correlated radiodensities along alongitudinal direction of the patient's tooth.

In certain embodiments, the shape data of the patient's crown isobtained via an impression, a plaster model or an intra-oral scan. Thevolumetric imagery is selected from a group consisting of tomographicimagery, ultrasonic imagery, cone beam computed tomography (CBCT)imagery and magnetic resonance imagery (MRI).

In further embodiments, the elements are vectors, and boundaries in theshape data correspond to the one or more crowns. The one or more crownsare represented by a plurality of limited length vectors and thevolumetric imagery is represented by a plurality of voxels.Intersections of the plurality of limited length vectors and theplurality of voxels are determined subsequent to the registering.

In further embodiments, the volumetric imagery is represented by a firstplurality of voxels, and the one or more crowns are represented by asecond plurality of voxels. The first plurality of voxels and the secondplurality of voxels are registered.

In further embodiments, one or more crowns are determined in the shapedata via segmentation of the shape data.

In yet further embodiments, the shape data is from intra-oral imagery,and the volumetric imagery is cone beam computed tomography (CBCT)imagery. The intra-oral imagery is of a higher precision than the CBCTimagery. The volumetric imagery includes both roots and crowns of teeth.The intra-oral imagery includes at least the crowns of the teeth butdoes not include an entirety of the roots of the teeth.

In still further embodiments, a determination is made of an area ofinterest in the intra-oral imagery, wherein the area of interestcorresponds to a location of the one or more crowns determined in theintra-oral imagery. An extraction is made within the volumetric imageryof the area of interest to reduce a size of the volumetric imagery.

Provided also are a method, system, and a computer readable storagemedium in which a computational device receives shape data of apatient's crown and volumetric imagery. A determination is made ofelements that represent one or more crowns in the shape data. Theelements are registered with corresponding voxels of the volumetricimagery. Volumetric coordinates and radiodensities are determined todetermine a tooth shape.

In additional embodiments, determining the tooth shape comprises fillingmissing or degraded data in the shape data.

In yet additional embodiments, determining the tooth shape comprisesfilling missing or degraded data in the volumetric imagery.

In further embodiments, the tooth shape is determined with greaterprecision in comparison to the received volumetric imagery, and thetooth shape is determined with greater precision with usage of lesserradiation. At least one of the patient's root is determined via regiongrowing from starting locations that include one or more of determinedvolumetric coordinates and radiodensities at the voxels.

In yet further embodiments, the volumetric imagery is represented by afirst plurality of voxels. The one or more crowns are represented byvectors or a second plurality of voxels. The first plurality of voxelsare registered to the vectors or the second plurality of voxels.

Provided also are a method, system, and a computer readable storagemedium in which for improving shape data of a patient's crown, the shapedata of the patient's crown is registered with volumetric data of thepatient's tooth.

Other aspects of the invention will become apparent by consideration ofthe detailed description and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

Referring now to the drawings in which like reference numbers representcorresponding parts throughout:

FIG. 1 illustrates a block diagram of a computing and imagingenvironment that includes a computational device that integratesintra-oral imagery and volumetric imagery, such as CBCT imagery, inaccordance with certain embodiments;

FIG. 2 illustrates a diagram in which an exemplary intra-oral imageryand advantages and disadvantages of intra-oral imagery are shown, inaccordance with certain embodiments;

FIG. 3 illustrates a diagram in which an exemplary CBCT imagery andadvantages and disadvantages of CBCT are shown, in accordance withcertain embodiments;

FIG. 4 illustrates a diagram that shows how an intra-oral imagery issegmented to determine crowns represented via limited length vectors, inaccordance with certain embodiments;

FIG. 5 illustrates a diagram that shows how the surface data obtainedvia intra-oral imagery may be represented via limited length vectors orvoxels, in accordance with certain embodiments;

FIG. 6 illustrates a diagram that shows how voxels represent CBCTimagery, in accordance with certain embodiments;

FIG. 7 illustrates a diagram that shows how the boundary between rootand crown is determined in CBCT imagery by integrating intra-oralimagery with CBCT imagery, in accordance with certain embodiments;

FIG. 8 illustrates a diagram that shows how surface data and volumetricdata are fitted to each other, in accordance with certain embodiments;

FIG. 9 illustrates a diagram that shows how surface data of the crown ismerged to volumetric data of the tooth, in accordance with certainembodiments;

FIG. 10 illustrates a diagram that shows characteristics of differenttypes of imagery, in accordance with certain embodiments;

FIG. 11 illustrates a diagram that shows how surface data extracted fromintra-oral imagery is fitted to model data maintained as a librarydataset;

FIG. 12 illustrates a flowchart for augmenting CBCT imagery with datafrom intra-oral imagery to determine boundary between roots and crowns,in accordance with certain embodiments;

FIG. 13 illustrates a flowchart for determining a localized area in CBCTimagery to generate a reduced size CBCT imagery, by augmenting CBCTimagery with data from intra-oral imagery, in accordance with certainembodiments;

FIG. 14 illustrates a diagram that shows how holes are filled inintra-oral imagery by integrating CBCT imagery with intra-oral imagery,in accordance with certain embodiments;

FIG. 15 illustrates a flowchart that shows how holes are filled inintra-oral imagery by integrating CBCT imagery with intra-oral imagery,in accordance with certain embodiments;

FIG. 16 illustrates a flowchart that shows how CBCT imagery isintegrated with intra-oral imagery, in accordance with certainembodiments;

FIG. 17 illustrates a block diagram that shows how limited lengthvectors of intra-oral imagery are registered to voxel data of CBCTimagery, in accordance with certain embodiments;

FIG. 18 illustrates a block diagram that shows how region growing isperformed to determine the entire tooth by following adjacent voxelswith correlated radiodensities at each and every intersecting voxelalong the direction of the centroid or any other longitudinal directionof a tooth, in accordance with certain embodiments;

FIG. 19 illustrates a flowchart that shows how the root of a tooth isbuilt from intersections of limited length vectors and voxels and regiongrowing, in accordance with certain embodiments; and

FIG. 20 illustrates a flowchart that shows how voxels of tomographyimagery and limited length vectors of shape data are integrated, inaccordance with certain embodiments;

FIG. 21 illustrates a flowchart that shows how missing or degraded datain shape data is filled by integrating voxels of tomography imagery andlimited length vectors of shape data, in accordance with certainembodiments;

FIG. 22 illustrates a flowchart that shows registration of elements inshape data with corresponding voxels in tomographic imagery to determinevolumetric coordinates and radiodensities at the voxels, in accordancewith certain embodiments;

FIG. 23 illustrates a flowchart that shows registration of elements inshape data of a patient's crown with corresponding voxels in volumetricimagery, in accordance with certain embodiments;

FIG. 24 illustrates a flowchart that shows registration of elements inshape data of a patient's crown with corresponding voxels in volumetricimagery to determine tooth shape, in accordance with certainembodiments; and

FIG. 25 illustrates a block diagram of a computational device that showscertain elements of the computational device shown in FIG. 1, inaccordance with certain embodiments.

DETAILED DESCRIPTION

In the following description, reference is made to the accompanyingdrawings which form a part hereof and which illustrate severalembodiments. It is understood that other embodiments may be utilized andstructural and operational changes may be made.

Intra-Oral Imagery and CBCT Imagery

Generally intra-oral images are of a significantly higher precision incomparison to CBCT images. Furthermore, CBCT data can be noisy. Also,the use of CBCT results in ionizing radiation to the patient and it isbest to use CBCT systems with as little radiation as possible.

In certain embodiments, a computational device receives shape data of apatient's crown and volumetric imagery of the patient's tooth. The shapedata may be generated from intra-oral images and may correspond to thesurface data of the patient's crown. The volumetric imagery may compriseCBCT imagery or other types of volumetric imagery. A determination ismade of voxels that represent one or more crowns in the shape data. Thevoxels in the shape data are registered with corresponding voxels of thevolumetric imagery.

In certain embodiments, segmented crowns determined from intra-oralimagery are registered to voxels of CBCT images. This allows moreaccurate determination of the boundary between the crown and the root ofa tooth in the CBCT data. It may be noted that without the use of theintra-oral imagery the boundary between the crown and the root of atooth may be fuzzy (i.e., not clear or indistinct) in CBCT imagery.

In certain embodiments, the surface scan data of an intra-oral imagingsystem is registered to the volumetric data obtained from a CBCT system.The 3-D coordinates of the crown boundaries that are found in theintra-oral imagery are mapped to the voxels of the CBCT imagery todetermine the boundary between roots and crowns at a sub-voxel levels ofaccuracy in the CBCT imagery. As a result, the roots can be extracted,even from noisy CBCT scan data.

In additional embodiments, holes in intra-oral imagery may be filled inby integrating CBCT imagery with intra-oral imagery.

Exemplary Embodiments

FIG. 1 illustrates a block diagram of a computing and imagingenvironment 100 that includes a computational device 102 that integratesintra-oral imagery 104 and CBCT imagery 106, in accordance with certainembodiments. The computational device 102 may include any suitablecomputational device such as a personal computer, a server computer, amini computer, a mainframe computer, a blade computer, a tabletcomputer, a touchscreen computing device, a telephony device, a cellphone, a mobile computational device, a dental equipment having aprocessor, etc., and in certain embodiments the computational device 102may provide web services or cloud computing services. In certainalternative embodiments, more than one computational device may be usedfor storing data or performing the operations performed by thecomputational device 102.

The intra-oral imagery 104 provides surface data of a patient's crownand the CBCT imagery 106 provides volumetric imagery of a patient'stooth, where the tooth may include both the crown and the root. Inalternative embodiments, the surface data of the patient's crown may beprovided by imagery that is different from intra-oral imagery, and thevolumetric imagery may be provided by other types of tomographicimagery, ultrasonic imagery, magnetic resonance imagery (MRI), etc. Thevolumetric imagery comprises three dimensional imagery and may berepresented via voxels.

The computational device 102 may include an integrating application 108,implemented in certain embodiments in software, hardware, firmware orany combination thereof. The integrating application 108 integrates theintra-oral imagery 104 and the CBCT imagery 106 to provide additionalfunctionalities that are not found in either the intra-oral imagery 104or the CBCT imagery 106 when they are not integrated.

The computational device 102 is coupled via one or more wired orwireless connections 110 to an intra-oral imaging system 112 and a CBCTimaging system 114, over a network 116. In certain embodiments, thenetwork 116 may comprise a local area network, the Internet, andintranet, a storage are network, or any other suitable network.

The intra-oral imaging system 112 may include a wand 117 having anintra-oral imaging sensor 118, where in certain embodiments theintra-oral imaging sensor 118 is an intra-oral camera that generatesintra-oral imagery of the oral cavity of a patient. The CBCT imagingsystem 114 may include a rotating X-ray equipment 120 that generatescross-sectional CBCT imagery of the soft tissue, hard tissue, teeth,etc. of a patient.

Therefore, FIG. 1 illustrates certain embodiments in which anintegrating application 108 that executes in the computational device102 integrates intra-oral imagery 104 generated by an intra-oral imagingsystem 112 with CBCT imagery 106 generated by a CBCT imaging system 114.In certain additional embodiments, the intra-oral imagery 104 and theCBCT imagery 106 may be stored in a storage medium (e.g., a disk drive,a floppy drive, a pen drive, a solid state device, an optical drive,etc.), and the storage medium may be coupled to the computational device102 for reading and processing by the integrating application 108.

FIG. 2 illustrates a diagram 200 in which an exemplary intra-oralimagery 202 is shown, in accordance with certain embodiments. Certainexemplary advantages 204 and certain exemplary disadvantages 206 of theintra-oral imagery 202 are also shown, in accordance with certainembodiments.

The intra-oral imagery 206 shows exemplary crowns (e.g., crowns 208 a,208 b, 208 c) in the upper arch of the oral cavity of a patient, wherethe intra-oral imagery 206 may have been acquired via the intra-oralimaging system 112. The crown is the portion of the tooth that may bevisually seen, and the root is the portion of the tooth that is hiddenunder the gum.

FIG. 2 shows that the intra-oral imagery is typically of a highprecision 210 in comparison with CBCT imagery. Additionally, noradiation that may cause harm to the patient (shown via referencenumeral 212) is needed in acquiring the intra-oral imagery 202. However,the intra-oral imagery 202 does not show the roots of teeth (referencenumeral 214) and may have holes 216, where a hole is a portion of thetooth that is not visible in intra-oral imagery. Holes may arise becauseof malocclusions or for other reasons. While, small and medium sizedholes may be filled (i.e., the hole is substituted via a simulatedsurface generated programmatically via the computational device 102) byanalyzing the intra-oral imagery 202, larger holes (i.e. holes thatexceed certain dimensions) may not be filled by just using data found inintra-oral imagery. Additionally, shiny surfaces of crowns may generatepoor quality intra-oral imagery (reference numeral 218).

Therefore, FIG. 2 illustrates certain embodiments in which intra-oralimagery may have holes and do not show the entirety of the roots ofteeth.

FIG. 3 illustrates a diagram 300 in which an exemplary CBCT imagery 302,and certain advantages 304 and certain disadvantages 306 of CBCT imageryare shown, in accordance with certain embodiments.

In the CBCT imagery the entire tooth (i.e., the root and the crown) isimaged (reference number 310) and there are few holes (reference number312). The few holes that exist may be caused by artifacts as a result ofamalgam fillings on tooth (reference numeral 320). However, the CBCTimages may be of a lower precision and may be more noisy in comparisonto intra-oral imagery (reference numeral 314). There is a potential forionizing radiation to the patient in the acquisition of CBCT imagery(reference numeral 316) unlike in intra-oral imagery in which there isno ionizing radiation in the acquisition process. Furthermore, while thecomplete tooth is imaged in CBCT imagery, the boundary between the rootand the crown may not be clear (reference numeral 318) as may be seen(reference numeral 320) in the exemplary CBCT imagery 302. The fuzzy andindistinct boundary 320 between the crown 322 and the root 324 may becaused by varying radiodensities during the process of acquiring CBCTimages. In certain embodiments, motion of the patient may generateinferior quality CBCT imagery.

Therefore, FIG. 3 illustrates certain embodiments in which CBCT imagesmay have low precision and have noisy data with the boundary between theroot and crown not being clearly demarcated.

FIG. 4 illustrates a diagram 400 that shows how an intra-oral imagery202 is segmented to determine crowns 402 represented via limited lengthvectors 404, in accordance with certain embodiments. The segmentation ofthe intra-oral imagery 202 to determine crowns 402 may be performed viathe integrating application 108 that executes in the computationaldevice 102. Exemplary segmented crowns are shown via reference numerals406 a, 406 b, 406 c. The segmented crowns are of a high resolution andshow clearly defined edges and are represented via limited lengthvectors 404. A vector has a direction and magnitude in three-dimensionalspace. A limited length vector is a vector whose length is limited. Inother embodiments, the segmented crowns may be represented via datastructures or mathematical representations that are different fromlimited length vectors 404.

Therefore, FIG. 4 illustrates certain embodiments in which intra-oralimagery is segmented to determine crowns represented via limited lengthvectors.

FIG. 5 illustrates a diagram that shows how an intra-oral imaging system410 scans the inside of a patient's mouth and generates surface samplesof the crowns of a patient's teeth, where the aggregated surface samplesmay be referred to as a point cloud 412.

The point cloud 412 may processed by the integrating application 108executing the computational device 102 to represent the surface of thecrowns. The crown of the tooth is a solid object, and the surfaces ofthe crown correspond to the boundaries of the solid object. The crownsurface may be represented by a surface mesh of node points connected bytriangles, quadrilaterals or via different types of polygon meshes. Inalternative embodiments, a solid mesh may also be used to represent thecrown surface. The process of creating the mesh is referred to astessellation.

In certain embodiments, the surface corresponding to the crown isrepresented in three dimensional space via limited length vectors 414 orvia voxels 416 or via other data structures 418. The voxels 416correspond to three-dimensional points on the surface of a crown. Incertain embodiments, the limited length vectors 414 may be converted tovowel representation via appropriate three dimensional coordinatetransformations 420. The limited length vectors 414 may correspond tothe sides of the different types of polygon meshes (e.g., triangles,quadrilaterals, etc.) in the surface representation of the crown.

Therefore, FIG. 5 illustrates certain embodiments in which intra-oralimagery is processed to determine crowns represented via limited lengthvectors or via voxels. The limited length vectors or voxels correspondto a surface data representation 422 of the crown. Surface data may alsobe referred to as shape data.

FIG. 6 illustrates a diagram 500 that shows how voxels 502 representCBCT imagery 302, in accordance with certain embodiments. A voxel (e.g.,voxel 504) is a volumetric pixel that is a digital representation ofradiodensity in a volumetric framework corresponding to the CBCT imagery302. The radio density may be measured in the Hounsfield scale. In FIG.6 an exemplary voxel representation 502 of part of the CBCT imagery 302is shown.

The voxel representation 502 has a local origin 504, with X, Y, Zcoordinates representing width, depth, and height respectively (shownvia reference numerals 506, 508, 510). The coordinate of the voxel wherethe X, Y, Z values are maximum are shown via the reference numeral 512.An exemplary voxel 504 and an illustrative column of voxels 514 are alsoshown. Each voxel has a volume defined by the dimensions shown viareference numerals 516, 518, 520.

In certain embodiments, limited length vectors of intra-oral imagery areregistered to the voxel representation of the CBCT imagery, to determinewhere the limited length vectors intersect the voxels of the CBCTimagery. In an exemplary embodiments, an intersecting limited lengthvector 522 is shown to intersect the voxels of the CBCT imagery atvarious voxels, wherein at least one voxel 524 at which the intersectiontakes place has a volumetric coordinate of (X,Y,Z) with an associatedradiodensity.

Therefore, FIG. 6 illustrates certain embodiments in which CBCT imageryis represented via voxels. The limited length vectors of the intra-oralimagery intersects the voxels of the CBCT imagery when both are placedin the same coordinate system, wherein each intersection has a X,Y,Zcoordinate and a radiodensity. In certain embodiments, the limitedlength vectors may be one or more of the sides of triangulatedtessellations used to represent shape data. The limited length vectorsmay be chained in shape representations.

FIG. 7 illustrates a diagram 600 that shows how the boundary betweenroot and crown is determined in CBCT imagery by integrating intra-oralimagery with CBCT imagery, in accordance with certain embodiments. Incertain embodiments, the voxel representation 606 of CBCT imagery isintegrated (via the integrating application 108) with the limited lengthvector representation or voxel representation 607 of the intra-oralimagery to overlay the high resolution clearly segmented crowns of theintra-oral imagery on the low resolution fuzzy crowns of the CBCTimagery (as shown via reference numeral 608), to clearly demarcate theboundary between roots and crowns in the CBCT imagery 602. In certainembodiments the integration of CBCT imagery and intra-oral imageryresults in a type of filtration operation that sharpens the CBCT imageryto determine the boundary between roots and crowns.

Therefore, FIG. 7 illustrates certain embodiments in which CBCT imageryis augmented with data from intra-oral imagery to determine the boundarybetween roots and crowns with a greater degree of accuracy in comparisonto using the CBCT imagery alone. As a result of the augmentation, highprecision crowns and low precision roots are obtained.

FIG. 8 illustrates a diagram 609 that shows how surface data andvolumetric data are fitted to each other, in accordance with certainembodiments. In certain embodiments, the surface data (i.e., the crownsurface data) may be represented with reference to a first coordinatesystem (shown via reference numeral 610) The volumetric data thatrepresents the tooth may be represented in a second coordinate system(shown via reference numeral 612).

In certain embodiments one or both of the crown surface data and thetooth volumetric data may have to be rotated 614, translated 616,morphed 618, scaled 620, or made to undergo other transformations 622 toappropriately overlap the crown surface data and the tooth volumetricdata in a single unified coordinate system. For example, in certainembodiments the tooth volumetric data is fitted to the crown surfacedata in the coordinate system of the tooth surface data by appropriaterotations, translations, morphing, scaling, etc., of the toothvolumetric data (as shown via reference numeral 624). In otherembodiments, crown surface data is fitted to the tooth volumetric datain the coordinate system of the tooth volumetric data by appropriaterotations, translations, morphing, scaling, etc., of the crown surfacedata (as shown via reference numeral 626). In other embodiments, boththe crown surface data and the tooth volumetric data may undergorotations, translations, morphing, scaling, etc. to fit the crownsurface data and tooth volumetric data in a new coordinate system (asshown via reference numeral 628).

FIG. 9 illustrates a diagram 650 that shows how surface data of thecrown is merged to volumetric data of the tooth, in accordance withcertain embodiments. An empty cube of voxels in the three dimensionalspace is populated with the shape data of a crown. As a result, thesurface data of the crown is represented via voxels of a threedimensional space 652.

The three dimensional space 652 with surface data is overlaid on thethree dimensional space 654 that has the volumetric representation ofthe tooth, to generate the overlay of the surface data on the volumetricdata shown in the three dimensional space 656. The fitting of thesurface data to the volumetric data may be performed via an iterativeclosest point (ICP) registration. ICP may fit points in surface data tothe points in volumetric data. In certain embodiment, the fitting mayminimize the sum of square errors with the closest volumetric datapoints and surface data points. In certain embodiments, the limitedlength vectors of the surface data are represented as voxels prior toperforming the ICP registration.

The anatomy of brackets, wires, filling or other features on the toothmay often assist in properly registering the surface data to thevolumetric data. The registration may in various embodiments beperformed via optimization techniques, such as simulated annealing,correlation techniques, dynamic programming, linear programming etc.

In certain embodiments a multiplicity of representations of the sameobject obtained by CBCT, magnetic resonance imagery (MRI), ultrasoundimagery, intra-oral imagery based surface data, etc., may be registeredto generate a better representation of a crown in comparison toembodiments that do not use data from the multiplicity ofrepresentations.

FIG. 10 illustrates a diagram 670 that shows characteristics ofdifferent types of imagery, in accordance with certain embodiments. Theintra-oral imagery 672 may provide not only the surface data 676 but mayalso be processed to provide information on reflectivity 678 andtranslucency 680 of the surface of the objects that are imaged. Forexample, the reflectivity and the translucency of the crown may bedifferent from that the gingiva, and the intra-oral imagery 672 may beprocessed to distinguish the crown from the gingiva based on thereflectivity and the translucency differences and the segmentation ofthe crown may be improved by incorporating such additional information.In certain embodiments where interferometry fringe patterns are used forcapturing the intra-oral imagery the reflectivity and translucencyinformation may be generated with greater precision in comparison toembodiments where such fringe patterns are not used.

In certain embodiments, the volumetric data 682 and the radiodensityinformation 684 corresponding to the CBCT imagery 674 may be used inassociation with the surface data 676, reflectivity information 678 andtranslucency information 680 of the intra-oral imagery 672 to provideadditional cues for performing the registration of the surface data 676and the volumetric data 682. Ray tracing mechanisms may also be used forsimulating a wide variety of optical effects, such as reflection andrefraction, scattering, and dispersion phenomena (such as chromaticaberration) for improving the quality of the different types of imagesand for registration.

FIG. 11 illustrates a diagram 688 that shows how surface data 690extracted from intra-oral imagery is fitted to one or more of model data694 a, 694 b, . . . 694 n maintained as a library dataset 692. Thelibrary dataset 692 may include model data for various types of teeth(e.g., incisors, canines, molars, etc.) and also model data for variouspatient parameters, such as those based on age, gender, ethnicity, etc.In certain embodiments where the CBCT imagery is unavailable, thesurface data 690 may be registered (reference numeral 696) to anappropriately selected model data 694 a . . . 694 n to provide betterquality information to a dental practitioner. When the roots of a toothare well formed and the crowns are relatively regular, then such fusionwith model data is often adequate for treatment purposes. However, withas little as two to three degrees of error in alignment, suchembodiments may have to be substituted with embodiments in which surfacedata from intra-oral imagery is registered with CBCT imagery to providebetter quality information to the dental practitioner. In certainadditional embodiments, the surface data is registered with the CBCTimagery with additional cues obtained from the model data.

FIG. 12 illustrates a flowchart 700 for augmenting CBCT imagery withdata from intra-oral imagery to determine the boundary between roots andcrowns, in accordance with certain embodiments. The operations shown inflowchart 700 may be performed via the integrating application 108 thatexecutes in the computational device 102.

Control starts at block 702 in which the computational device 102receives intra-oral imagery 104 and CBCT imagery 106. The integratingapplication 108 determines (at block 704) one or more crowns in theintra-oral imagery, wherein the one or more crowns of the intra-oralimagery are represented by limited length vectors or voxels, and theCBCT imagery is represented by voxels. Control proceeds to block 706, inwhich the integrating application 108 integrates the one or more crownsdetermined in the intra-oral imagery into the CBCT imagery byregistering the limited length vectors or voxels that represent the oneor more crowns in the intra-oral imagery with the voxels of the CBCTimagery, to determine a boundary between at least one crown and at leastone root in the CBCT imagery.

FIG. 13 illustrates a flowchart 800 for determining a localized area inCBCT imagery to generate a reduced size CBCT imagery, by augmenting CBCTimagery with data from intra-oral imagery, in accordance with certainembodiments. The operations shown in flowchart 800 may be performed viathe integrating application 108 that executes in the computationaldevice 102.

Control starts at blocks 802 and 804 in which CBCT imagery andintra-oral imagery are provided to the integrating application 108. Theintegrating application 108 determines (at block 806) an area ofinterest in the intra-oral imagery, wherein the area of interestcorresponds to a location of the one or more crowns determined in theintra-oral imagery via segmentation.

Control proceeds to block 808 in which the integrating application 108extracts from the CBCT imagery the area of interest to reduce the sizeof the CBCT imagery, and the reduced size CBCT imagery is stored (atblock 810) in the computational device 102.

Therefore FIG. 8 illustrates certain embodiments in which the size ofCBCT imagery is reduced by incorporating an area of interest determinedfrom intra-oral imagery.

FIG. 14 illustrates a diagram 900 that shows how holes are filled inintra-oral imagery by integrating CBCT imagery with intra-oral imagery,accordance with certain embodiments.

In FIG. 9 an exemplary intra-oral imagery 104 has holes 902 (i.e., areasof the crown of teeth that are not imaged by the intra-oral imagingsystem 112). The integrating application 108 uses the CBCT imagery 106to fill the holes via the low precision crowns without holes that arefound in the CBCT imagery 106, to generate augmented intra-oral imagingdata 904 in which all holes are filled. In certain embodiments, a rangeof radiodensities are determined in voxels of a determined boundarybetween roots and crowns, and based on the range of radiodensities andthe determined boundary, the holes in the intra-oral imagery are filledfrom selected voxels of the CBCT imagery.

FIG. 15 illustrates a flowchart 1000 that shows how holes are filled inintra-oral imagery by integrating CBCT imagery with intra-oral imagery,accordance with certain embodiments. The operations shown in flowchart1000 may be performed via the integrating application 108 that executesin the computational device 102.

Control starts at block 1002 in which the computational device 102receives intra-oral imagery 104 and volumetric imagery, such as, conebeam computed tomography (CBCT) imagery 106. Control proceeds to block1004, in which the integrating application 108 determines one or morecrowns in the intra-oral imagery 104 and the CBCT imagery 106, where theone or more crowns determined by the intra-oral imagery 104 has one ormore holes, and where a hole is a part of a tooth that is not visible inthe intra-oral imagery. The one or more crowns determined in the CBCTimagery are integrated (at block 1006) into the intra-oral imagery 104,to fill the one or more holes in the intra-oral imagery.

Therefore FIGS. 14 and 15 illustrate how holes are filled in intra-oralimagery by integrating information from CBCT imagery. Conversely, ifmissing or degraded data is found in volumetric imagery, such missing ordegraded data may be filled from surface data found in the intra-oralimagery.

FIG. 16 illustrates a flowchart 1100 that shows how CBCT imagery 106 isintegrated with intra-oral imagery 104, in accordance with certainembodiments. The operations shown in flowchart 1100 may be performed viathe integrating application 108 that executes in the computationaldevice 102.

Control starts at block 1102 in which a computational device 102receives intra-oral imagery 104 and CBCT imagery 106. The intra-oralimagery 104 and the CBCT imagery 106 are integrated (at block 1104), todetermine a boundary between at least one crown and at least one root inthe CBCT imagery 106, and to fill one or more holes in the intra-oralimagery 104.

FIG. 17 illustrates a block diagram 1200 that shows how limited lengthvectors of intra-oral imagery are registered to voxel data of CBCT orother volumetric imagery, in accordance with certain embodiments.

In FIG. 17 the hatched area indicated via reference numeral 1202indicates an uncertainty region of the CBCT imagery in which the actualtooth boundary of the patient is likely to found. The limited lengthvectors (or voxels) of the intra-oral imagery are registered to thevoxels of the CBCT imagery to determine the intersections 1204. At eachof the intersections 1204 there is an X,Y,Z coordinate and an associatedradiodensity (shown via reference numeral 1206), where adjacent voxelsmay have similar radiodensities or correlated radiodensities in theuncertainty region 1202 (as shown via reference numeral 1208).

FIG. 18 illustrates a block diagram 1300 that shows how region growingis performed to determine the entire tooth by following adjacent voxelswith correlated radiodensities at each and every intersecting voxelalong the direction of the centroid 1302 of a tooth, in accordance withcertain embodiments. The centroid is located along a longitudinaldirection of the tooth. The correlated radiodensities may be determinedvia correlation windows of different sizes. For example, a cube ofvoxels with length, breadth, and height of three voxels each may be usedas a correlation window to determine which adjacent voxel is mostcorrelated to a previously determined voxel in terms of radiodensities.

Reference numeral 1306 shows the entire tooth outlined via regiongrowing with seed values starting from the voxels and limited lengthvector (or surface voxel) intersections 1204 and the associatedradiodensities. Other mechanisms may also be adopted for region growingto determine the entire tooth.

FIG. 19 illustrates a flowchart 1400 that shows how the root of a toothis built from intersections of limited length vectors (or surface voxel)and voxels and region growing, in accordance with certain embodiments.Control starts at block 1402 where the voxel information at each voxelof a CBCT image is given by a volumetric coordinate X,Y,Z and theradiodensity. Control proceeds to block 1404 in which a determination ismade as to which voxels of CBCT image and limited length vectors (orvoxel) of the boundary of the crown of intra-oral image intersect. Theroot of the tooth is built (at block 1406) from the determinedintersections via region growing techniques based on following adjacentradiodensities that are correlated (i.e., similar in magnitude) to eachother.

FIG. 20 illustrates a flowchart 1500 that shows how voxels of tomography(i.e. volumetric) imagery and limited length vectors of shape data areintegrated, in accordance with certain embodiments. A computationaldevice receives (at block 1502) shape data of a patient's dentition andtomography imagery. Vectors that represent one or more crowns in theshape data are determined (at block 1504). The vectors are registeredwith corresponding voxels of the tomography imagery, and volumetriccoordinates and radiodensities at the voxels are determined (at block1506). At least one of the patient's teeth is determined via regiongrowing from starting locations that include one or more of thedetermined volumetric coordinates and the radiodensities at the voxels,and the region growing is performed by following adjacent voxels withclosest radiodensities along a direction of a centroid of a tooth (atblock 1508). In alternative embodiments voxels (referred to as surfacevoxel) corresponding to the limited length vectors of the surface datamay be used instead of the limited length vectors for registration.

FIG. 21 illustrates a flowchart 1600 that shows how missing or degradeddata in shape data is filled by integrating voxels of tomography imageryand limited length vectors of shape data, in accordance with certainembodiments. A computational device receives (at block 1602) shape dataof a patient's dentition and tomography imagery. Vectors that representone or more crowns in the shape data are determined, wherein the one ormore crowns has degraded data or missing data (at block 1604). Thevectors are registered with corresponding voxels of the tomographyimagery, and volumetric coordinates and radiodensities at the voxels aredetermined (at block 1606). At least one of the patient's teeth isdetermined via region growing from starting locations that include oneor more of the determined volumetric coordinates and the radiodensitiesat the voxels to fill the degraded or the missing data in the one ormore crowns of the shape data (at block 1606).

In certain alternative embodiments vectors are registered withcorresponding voxels of the tomography imagery to determine volumetriccoordinates and radiodensities at the voxels, to determine a tooth withgreater precision and to fill missing or degraded data in the shapedata. In certain embodiments, by determining the tooth with greaterprecision the received tomography imagery is obtained with usage oflesser radiation.

FIG. 22 illustrates a flowchart 1700 that shows registration of elements(e.g., vectors) in shape data with corresponding voxels in tomographicimagery to determine volumetric coordinates and radiodensities at thevoxels, in accordance with certain embodiments. A computational devicereceives (at block 1702) shape data of a patient's dentition andtomography imagery. Elements (e.g. vectors or voxels) that represent oneor more boundaries in the shape data are determined (at block 1704). Theelements are registered with corresponding voxels of the tomographyimagery, and volumetric coordinates and radiodensities at the voxels aredetermined (at block 1706). In certain embodiments, the boundaries inthe shape data delineate one or more crowns of teeth.

FIG. 23 illustrates a flowchart 2300 that shows registration of elementsin shape data of a patient's crown with corresponding voxels involumetric imagery, in accordance with certain embodiments.

Control starts at block 2302 in which shape data of a patient's crownand volumetric imagery of the patient's tooth is received. Adetermination is made (at block 2304) of elements that represent one ormore crowns in the shape data. A computational device is used toregister (at block 2306) the elements with corresponding voxels of thevolumetric imagery.

FIG. 24 illustrates a flowchart 2400 that shows registration of elementsin shape data of a patient's crown with corresponding voxels involumetric imagery to determine tooth shape, in accordance with certainembodiments.

Control starts at block 2402 in which shape data of a patient's crownand volumetric imagery are received. A determination is made (at block2404) of elements that represent one or more crowns in the shape data.The elements are registered (at block 2406) with corresponding voxels ofthe volumetric imagery by using a computational device, and volumetriccoordinates and radiodensities are determined to determine a toothshape.

Therefore, FIGS. 1-24 illustrate certain embodiments in which the toothof a patient is determined more accurately by integrating informationextracted from intra-oral imagery and CBCT imagery. Also, degraded ormissing data in the crowns of intra-oral imagery are filled byintegrating information extracted from CBCT imagery. By integratingintra-oral imagery with CBCT imagery, both intra-oral imagery and CBCTimagery are enhanced to have greater functionalities and CBCT imagerymay be obtained with usage of a lower amount of radiation.

Further Details of Embodiments

In a volumetric data representation there may be areas of high contrastand low contrast. When segmenting via thresholding (e.g., bythresholding radiodensities) it may be easier to threshold crowns thanroots. This is because crowns appear with high density against softtissue. It may be noted that roots appear with low contrast against thebone. High contrast junctions may be easier to segment this manner. Incertain embodiments, the crowns may be thresholded and the borders maybe used to seed the segmentation to isolate the roots. Thus thevolumetric data set may be used to segment itself. This mayautomatically register the crown root object. This may even be used toregister the crown surface data.

In certain embodiments, instead of segmenting roots, certain embodimentsmay extract only the centroid of the root.

Certain embodiments may link the shape and tomography imagery datatogether in a file system. For example, information may be added to theheaders of the image files of both the CBCT and intra-oral scan data toenable viewing software to easily reference one from the other.Alternatively, the viewing software may keep track of which intra-oralscan image and CBCT image files have been registered with one anotherand store the information in a separate file. In certain embodimentscorrelation or optimization techniques may be used to find theintersection points in the image data.

In certain embodiments, the output of the processes is a data structurethat is an advanced representation of the surface or a volumetric dataenhanced by the fusion process of registration of multiple sources ofimagery. Multidimensional data representation and visualizationtechniques may be used to display such enhanced surfaces or volumes. Incertain embodiments, the collected image data may after processing andregistration be rendered and displayed as three dimensional objects viavolumetric rendering and segmentation.

Additional Details of Embodiments

The operations described in the figures may be implemented as a method,apparatus or computer program product using techniques to producesoftware, firmware, hardware, or any combination thereof. Additionally,certain embodiments may take the form of a computer program productembodied in one or more computer readable storage medium(s) havingcomputer readable program code embodied therein.

A computer readable storage medium may include an electronic, magnetic,optical, electromagnetic, or semiconductor system, apparatus, or device,or any suitable combination of the foregoing. The computer readablestorage medium may also comprise an electrical connection having one ormore wires, a portable computer diskette or disk, a hard disk, a randomaccess memory (RAM), a read-only memory (ROM), an erasable programmableread-only memory (EPROM or Flash memory), a portable compact discread-only memory (CD-ROM), an optical storage device, a magnetic storagedevice, etc. A computer readable storage medium may be any tangiblemedium that can contain, or store a program for use by or in connectionwith an instruction execution system, apparatus, or device.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages.

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, system andcomputer program products according to certain embodiments. At leastcertain operations that may have been illustrated in the figures showcertain events occurring in a certain order. In alternative embodiments,certain operations may be performed in a different order, modified orremoved. Additionally, operations may be added to the above describedlogic and still conform to the described embodiments. Further,operations described herein may occur sequentially or certain operationsmay be processed in parallel. Yet further, operations may be performedby a single processing unit or by distributed processing units. Computerprogram instructions can implement the blocks of the flowchart. Thesecomputer program instructions may be provided to a processor of acomputer for execution.

FIG. 25 illustrates a block diagram that shows certain elements that maybe included in the computational device 102, in accordance with certainembodiments. The system 2500 may comprise the computational device 102and may include a circuitry 2502 that may in certain embodiments includeat least a processor 2504. The system 2500 may also include a memory2506 (e.g., a volatile memory device), and storage 2508. The storage2508 may include a non-volatile memory device (e.g., EEPROM, ROM, PROM,RAM, DRAM, SRAM, flash, firmware, programmable logic, etc.), magneticdisk drive, optical disk drive, tape drive, etc. The storage 2508 maycomprise an internal storage device, an attached storage device and/or anetwork accessible storage device. The system 2500 may include a programlogic 2510 including code 2512 that may be loaded into the memory 2506and executed by the processor 2504 or circuitry 2502. In certainembodiments, the program logic 2510 including code 2512 may be stored inthe storage 2508. In certain other embodiments, the program logic 2510may be implemented in the circuitry 2502. Therefore, while FIG. 25 showsthe program logic 2510 separately from the other elements, the programlogic 2510 may be implemented in the memory 2506 and/or the circuitry2502.

The terms “an embodiment”, “embodiment”, “embodiments”, “theembodiment”, “the embodiments”, “one or more embodiments”, “someembodiments”, and “one embodiment” mean “one or more (but not all)embodiments of the present invention(s)” unless expressly specifiedotherwise.

The terms “including”, “comprising”, “having” and variations thereofmean “including but not limited to”, unless expressly specifiedotherwise.

The enumerated listing of items does not imply that any or all of theitems are mutually exclusive, unless expressly specified otherwise.

The terms “a”, “an” and “the” mean “one or more”, unless expresslyspecified otherwise.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. In addition, devices that are in communication with eachother may communicate directly or indirectly through one or moreintermediaries.

A description of an embodiment with several components in communicationwith each other does not imply that all such components are required. Onthe contrary a variety of optional components are described toillustrate the wide variety of possible embodiments.

When a single device or article is described herein, it will be readilyapparent that more than one device/article (whether or not theycooperate) may be used in place of a single device/article. Similarly,where more than one device or article is described herein (whether ornot they cooperate), it will be readily apparent that a singledevice/article may be used in place of the more than one device orarticle or a different number of devices/articles may be used instead ofthe shown number of devices or programs. The functionality and/or thefeatures of a device may be alternatively embodied by one or more otherdevices which are not explicitly described as having suchfunctionality/features.

The foregoing description of various embodiments of the invention hasbeen presented for the purposes of illustration and description. It isnot intended to be exhaustive or to limit the invention to the preciseform disclosed. Many modifications and variations are possible in lightof the above teaching. It is intended that the scope of the invention belimited not by this detailed description, but rather by the claimsappended hereto. The above specification, examples and data provide acomplete description of the manufacture and use of the composition ofthe invention. Since many embodiments of the invention can be madewithout departing from the spirit and scope of the invention, theinvention resides in the claims hereinafter appended.

What is claimed is:
 1. A method comprising: receiving, by a computer,shape data of a super-gingival portion of a patient's first tooth;receiving, by the computer, volumetric imagery of the super-gingivalportion of the first tooth and volumetric imagery of a sub-gingivalportion of the first tooth; registering the shape data of thesuper-gingival portion with the volumetric data of the super-gingivalportion to obtain a registration result using the computer; determining,by the computer, at least one criterion for detecting a surface of thefirst tooth in the volumetric imagery of the super-gingival or thesub-gingival portions using the registration result; and detecting, bythe computer, a surface of the sub-gingival portion of the first toothwithin the volumetric imagery of the sub-gingival portion of the firsttooth using the at least one criterion, wherein determining the at leastone criterion includes determining coordinate information for thevolumetric imagery registered with shape data corresponding to thesuper-gingival portion of the first tooth, and wherein detecting asurface of the sub-gingival portion of the first tooth within thevolumetric imagery includes identifying a portion of the volumetricimagery corresponding to the sub-gingival portion of the first tooththat is adjacent to a portion of the volumetric imagery registered withthe shape data corresponding to the super-gingival portion of the firsttooth using the coordinate information.
 2. The method of claim 1,further comprising: receiving, by the computer, shape data ofsuper-gingival portions of a plurality of teeth; receiving, by thecomputer, volumetric imagery of super-gingival and sub-gingival portionsof the plurality of teeth; and determining, by the computer, elementsthat represent the super-gingival portions of the plurality of teethwithin the shape data.
 3. The method of claim 2, further comprising:registering, by the computer, the shape data of the super-gingivalportions of the plurality of teeth with the volumetric data of thesuper-gingival portions of the plurality of teeth to obtain theregistration result; and detecting, by the computer, surfaces of thesub-gingival portions of the plurality of teeth within the volumetricimagery of the sub-gingival portions of the plurality of teeth using theat least one criterion.
 4. The method of claim 1, wherein determining atleast one criterion for detecting a surface of the first tooth in thevolumetric imagery includes determining radiodensities corresponding toportions of the volumetric imagery registered with the shape data. 5.The method of claim 4, wherein detecting a surface of the sub-gingivalportion of the first tooth includes identifying adjacent portions thatpossess correlated radiodensities along a longitudinal direction of thepatient's first tooth.
 6. The method of claim 5, wherein detecting asurface of the sub-gingival portion of the first tooth within thevolumetric imagery includes comparing a radiodensity of the firstportion of the volumetric imagery to a radiodensity of a second portionof the volumetric imagery.
 7. The method of claim 1, further comprisingcapturing the volumetric imagery using an imaging modality selected froma group consisting of tomographic imagery, ultrasonic imagery, cone beamcomputed tomography, and magnetic resonance imagery.
 8. A dental imagingsystem for identifying a sub-gingival surface of a tooth in volumetricimagery data, the dental imaging system including a processor and amemory, the memory storing instructions that, when executed by theprocessor, cause the dental imaging system to: receive shape data of asuper-gingival portion of a patient's first tooth; receive volumetricimagery data of the super-gingival portion of the first tooth andvolumetric imagery data of a sub-gingival portion of the first tooth;register the shape data of the super-gingival portion with thevolumetric imagery data of the super-gingival portion to obtain aregistration result; determine at least one criterion for detecting asurface of the first tooth in the volumetric imagery data of thesuper-gingival or the sub-gingival portions using the registrationresult; and detect a surface of the sub-gingival portion of the firsttooth within the volumetric imagery data of the sub-gingival portion ofthe first tooth using the at least one criterion, wherein theinstructions, when executed by the processor, cause the dental imagingsystem to determine the at least one criterion by determining coordinateinformation for portions of the volumetric imagery registered with shapedata corresponding to the super-gingival portion of the first tooth, andto detect the surface of the sub-gingival portion of the first toothwithin the volumetric imagery by identifying a portion in the volumetricimagery corresponding to the sub-gingival portion of the first tooththat is adjacent to a portion of the volumetric imagery registered withthe shape data corresponding to the super-gingival portion of the firsttooth using the coordinate information.
 9. The dental imaging system ofclaim 8, wherein the instructions, when executed by the processor,further cause the dental imaging system to: receive shape data ofsuper-gingival portions of a plurality of teeth; receive volumetricimagery data of super-gingival and sub-gingival portions of theplurality of teeth; and determine elements that represent thesuper-gingival portions of the plurality of teeth within the shape data.10. The dental imaging system of claim 9, wherein the instructions, whenexecuted by the processor, further cause the dental imaging system to:register the shape data of the super-gingival portions of the pluralityof teeth with the volumetric imagery data of the super-gingival portionsof the plurality of teeth to obtain the registration result; and detectsurfaces of the sub-gingival portions of the plurality of teeth withinthe volumetric imagery data of the sub-gingival portions of theplurality of teeth using the at least one criterion.
 11. The dentalimaging system of claim 8, wherein the instructions, when executed bythe processor, cause the dental imaging system to determine the at leastone criterion for detecting the surface of the first tooth in thevolumetric imagery data by determining radiodensities corresponding toportions of the volumetric imagery data registered with the shape data.12. The dental imaging system of claim 11, wherein the instructions,when executed by the processor, cause the dental imaging system todetect the surface of the sub-gingival portion of the first tooth in thevolumetric imagery data by identifying adjacent portions that possesscorrelated radiodensities along a longitudinal direction of thepatient's first tooth.
 13. The dental imaging system of claim 12,wherein the instructions, when executed by the processor, cause thedental imaging system to detect a surface of the sub-gingival portion ofthe first tooth within the volumetric imagery data by comparing aradiodensity of a first portion of the volumetric imagery to aradiodensity of a second portion of the volumetric imagery.
 14. Thedental imaging system of claim 8, further comprising an imaging deviceconfigured to capture the volumetric imagery data using an imagingmodality selected from a group consisting of tomographic imagery,ultrasonic imagery, cone beam computed tomography, and magneticresonance imagery, and wherein the instructions, when executed by theprocessor, cause the dental imaging system to receive the volumetricimagery data by receiving the volumetric imagery from the imagingdevice.
 15. A dental imaging system for identifying a sub-gingivalsurface of a tooth in volumetric imagery data, the dental imaging systemcomprising: a surface scanner configured to capture shape data of asuper-gingival portion of a patient's first tooth; a volumetric imagingdevice configured to capture volumetric imagery data of thesuper-gingival portion of the first tooth and volumetric imagery data ofa sub-gingival portion of the first tooth; and a computer configured toreceive the shape data from the surface scanner, receive the volumetricimagery data of the super-gingival portion of the first tooth and thevolumetric imagery data of the sub-gingival portion of the first toothfrom the volumetric imaging device, register the shape data of thesuper-gingival portion with the volumetric imagery data of thesuper-gingival portion to obtain a registration result, determinecoordinate information for the volumetric imagery registered with shapedata corresponding to the super-gingival portion of the first tooth, anddetect a surface of the sub-gingival portion of the first tooth withinthe volumetric imagery data of the sub-gingival portion of the firsttooth by identifying a portion of the volumetric imagery correspondingto the sub-gingival portion of the first tooth that is adjacent to aportion of the volumetric imagery registered with the shape datacorresponding to the super-gingival portion of the first tooth using thecoordinate information.